1. Field of the Invention
The field of the invention is memory systems for automated computing machinery.
2. Description of Related Art
The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely complicated devices. Today's computers are much more sophisticated than early systems such as the EDVAC. Computer systems typically include a combination of hardware and software components, application programs, operating systems, processors, buses, memory, input/output devices, and so on. As advances in semiconductor processing and computer architecture push the performance of the computer higher and higher, more sophisticated computer software has evolved to take advantage of the higher performance of the hardware, resulting in computer systems today that are much more powerful than just a few years ago.
Contemporary high performance computing main memory systems incorporate memory devices in an array of dynamic random access memory (‘DRAM’) devices. FIG. 1 illustrates an example of a prior art memory system that includes a memory controller (102), memory modules (106, 108, 110), memory hub devices (118, 120, 122), and memory devices (112, 114, 116) organized in a hub-and-spoke topology with the memory hub devices as ‘hubs’ and the memory devices as the ‘spokes.’ The memory controller (102) is interconnected to a system processor (104) through a physical, high speed frontside bus (176), and the memory controller (102) is interconnected to the memory hub devices (118, 120, 122) through a physical high speed memory bus (136) and bus stubs (124, 126, 128). The high speed memory bus (136) is terminated in its characteristic impedance by memory bus terminator (130). Each memory hub device provides one or more lower speed independent connections to banks of memory devices (112, 114, 116). The example of FIG. 1 illustrates only one ‘channel’ (134) or network of memory hub devices connected by a bus and bus stubs to a memory controller. Practical memory systems, however, typically may be implemented with additional such channels or networks (138) as well.
In the example of FIG. 1, the high speed memory bus (136) is a set of parallel conductive pathways that conduct memory signals at very high frequencies, often in excess of a gigahertz. Such a high speed memory bus functions as a transmission line or a group of transmission lines. A bus stub is an additional section of such transmission line, connected in parallel to the main line, that connects the bus proper to a memory hub device. A bus stub may be connected between the main bus and a memory hub device by soldered interconnects, connectors, pressure contacts, conductive adhesives, optical interconnects, and other communication and power delivery methods. A high speed memory bus, like any transmission line, has a characteristic impedance at its typical operating frequency or range of operating frequencies. A high speed memory bus, like any transmission line, suffers from noise and crosstalk and requires power to drive signals on the bus. Terminating the high memory bus in a memory bus terminator (130) having the same impedance as the characteristic impedance of the high speed memory bus helps to minimize noise and crosstalk on the bus and helps to minimize the power requirements for driving memory signals on the bus.
All connections to bus stubs affect the characteristic impedance of the high speed memory bus at the point of the connection, causing reflections of part of the signals on the bus, increasing bus noise, increasing crosstalk among conductive pathways of the bus, and increasing power requirements to drive signals on the bus. As computer technology advances, memory requirements and memory speed requirements are more and more demanding. Memory bus frequencies climb higher and higher, and the use of memory architectures with bus stubs like the one illustrated in FIG. 1 become more and more difficult. The benefit of an architecture like the one in FIG. 1 is flexibility. The memory system of FIG. 1 is illustrated with only three bus stubs and only three memory modules. As a practical matter, however, such a system may be implemented with any number of physical connectors so that users can insert or remove memory modules as desired. Of course, each such insertion or removal affects the bus impedance and the noise, crosstalk, and power requirements of the bus. As memory bus frequencies increase, an architecture like the one in FIG. 1 eventually becomes impractical, despite its benefits.
FIG. 2 illustrates an example of an integrated circuit of a kind that was used to implement a prior art memory hub device (118). The integrated circuit of FIG. 2 is implemented as an integrated circuit die (100) encapsulated in an integrated circuit package (136). Conductive pathways (138, 140, 142, 144) of a high speed memory bus are brought in to the integrated circuit package (136) die through package pins (137) as part of a bus stub connecting the circuit of FIG. 2 to a high speed memory bus. Conductive pathways (148, 150, 152, 154) inside the die are connected to conductive pathways of the bus outside the die (but still inside the integrated circuit package) through die pads (146). The conductive pathways inside the die connect bus signals to logic circuitry (170, 174) of the die through communications circuits (158, 164). In this example, even the small conductive pathways inside the die represent disruptions of the bus impedance that increase bus noise, crosstalk, and power requirements. Even if it is only the small conductive pathways inside the die itself that present the stub effect disrupting bus impedance, use of such an architecture becomes impractical at high bus speeds.
FIG. 3 illustrates a prior art memory architecture with cascaded memory hub devices, an architecture that was developed at least partly to try to overcome the problems with memory bus stubs. The memory controller (102) is interconnected to a system processor (104) through a physical, high speed frontside bus (176). Rather than connecting memory modules with multiple connections to a memory bus through bus stubs that affect bus impedance, however, each memory hub device (118, 120, 122) in the system of FIG. 3 is connected to just one other device, either the memory controller (102) or another memory hub device, through a single point to point link (178, 180, 182). Like a memory bus, a link is a set of parallel conductive pathways for conducting memory signals between a memory controller and a memory module, or between memory modules. Unlike a bus, a link makes only one such connection, well terminated on each end in the characteristic impedance of the link. No stubs are attached, so no stub can affect the impedance of the link.
Memory access latency in memory system network topologies composed of cascaded memory hub devices together with point-to-point electrical or optical links is degraded by having to propagate through each memory hub device in a cascaded network of memory hub devices between a particular rank of memory devices and a memory controller. Communications of signals in this network are synchronized, typically with respect to a memory bus clock signal. Memory signals, both instructions and data to be written or read, outbound from the memory controller are cached at least briefly in each memory hub device, resynchronized, and retransmitted on the next outbound link. Inbound memory signals also are cached, resynchronized, and retransmitted until they arrive in the memory controller. Take, for example, the transition delay for signal transition across a memory hub device as one unit of latency. The latency for transmission of memory signals between memory controller (102) and memory devices (112) served by memory hub device (118) may be said to have a value of 1. The latency for transmission of memory signals between memory controller (102) and memory devices (114) served by memory hub device (120) has a value of 2. And so on. Even though memory signals are resynchronized and retransmitted as fast as possible, latency always varies across ranks of memory served by the various memory hub devices in such an architecture.
In addition to issues with memory access latency, power consumption and cost for interfaces are both maximized by having point-to-point network connections. In the memory system of FIG. 1, for example, the memory controller and each memory hub device drives output memory signals across the full width of each link. If there are 10 lines in an outbound link, the memory controller or memory hub device driving the link has 10 output drivers in its output interface to the link with a power requirement to supply all 10 drivers. For all these reasons, there is an ongoing need for innovation in the field of memory systems.